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Abstract — In this paper, we present coherence-based per- 
formance guarantees of Orthogonal Matching Pursuit (OMP) 
for both support recovery and signal reconstruction of sparse 
signals when the measurements are corrupted by noise. In 
particular, two variants of OMP either with known sparsity 
level or with a stopping rule are analyzed. It is shown that if the 
measurement matrix X 6 c nxp satisfies the strong coherence 
property, then with n > 0(k logp), OMP will recover a fc-sparse 
signal with high probability. In particular, the performance 
guarantees obtained here separate the properties required of 
the measurement matrix from the properties required of the 
signal, which depends critically on the minimum signal to noise 
ratio rather than the power profiles of the signal. We also 
provide performance guarantees for partial support recovery. 
Comparisons are given with other performance guarantees 
for OMP using worst-case analysis and the sorted one step 
thresholding algorithm. 

Index Terms — Compressive Sensing (CS), Orthogonal Match- 
ing Pursuit (OMP), worst-case coherence, average coherence, 
support recovery, signal reconstruction 

I. Introduction 

Sparse signal processing is a fundamental task in many 
applications involving high-dimensional data. Many of the 
recent advances in Compressive Sensing (CS) (TJ, have 
been centered around reconstructing a sparse signal j3 € C p 
with a few nonzero entries, from a number of linear mea- 
surements that is much smaller than the signal dimension, 
possibly corrupted by noise r/, given as y = X (3 + rj, where 
X G C«xp is the measurement matrix. Two main classes 
of algorithms have been successfully applied, one is convex 
optimization based algorithms such as Basis Pursuit (BP) 0, 
the other one is greedy pursuit based algorithms such as 
Orthogonal Matching Pursuit (OMP) 0. In particular, the 
latter class is appealing and competitive in practice due to its 
simplicity and low computational cost 0. 

The performance of OMP can be characterized either in 
a worst-case sense or in an average (probabilistic) sense. 
Define the worst coherence of the measurement matrix X as 
fx = max^j \ (xi,Xj)\, where a;, denotes the ith unit-norm 
column of X. It is shown in that if \i < then OMP 

recovers any fc-sparse vector j3 from the noiseless measure- 
ment y — Xf3, and this result is confirmed to be sharp in 



0. It is further studied in |7| that given the amplitudes of 
the nonzero entries of j3 are not too small, OMP recovers 
the support of the signal from noisy observations. From the 
Welch bound which gives /it > C(n _1 / 2 ), in order to 
recover all fc-sparse vectors, the number of measurements 
is required to satisfy n > (D(k 2 ). It is demonstrated in 
that, if the Restricted Isometry Property (RIP) of X of 
order fc + 1 is satisfied with constant smaller than l/(3vfc), 
then OMP recovers any fc-sparse signal, again from the 
noiseless measurements y = X(3. Given a random matrix 
satisfies the RIP of order fc with constant 5k provided that 
n > O (k \og(p / k) / 5%) , then the number of measurements is 
required n > 0(k 2 log(p/fc)). These results all suffer from 
what is called "square-root bottleneck". Alternatively, instead 
of aiming to recover all fc sparse signals using OMP, it is 
shown that for a fixed sparse vector f3, a randomly drawn 
measurement matrix X from i.i.d. normal entries can recover 
(3 with high probability with n > O(fclogp) measurements 
from the noiseless measurements y = X/3, However, it 
does not provide a practical way to design or validate the 
usefulness of a measurement matrix. 

In this paper, we aim to use the strong coherence proper- 
ties proposed in ffTOl to capture the performance of OMP. 
The strong coherence properties require that the worst- 
case coherence of the measurement matrix X is sufficiently 
small, and also that the average coherence, defined as v = 
— 3j max, I ^2j-jjti(xi,Xj)\, is small relative to the worst- 
case coherence. Different from the worst-case sense, we aim 
to provide conditions on the measurement matrix X that 
are easily verifiable in contrast to RIP, and that succeeds 
in support recovery and reconstruction of sparse signals 
with high probability in the presence of noise. We show 
that if X satisfied the strong coherence property, then with 
n > O(klogp) measurements, OMP recovers a fc-sparse 
signal with high probability. In particular, the performance of 
OMP depends on the smallest signal-to-noise ratio SNR min 
determined by the smallest nonzero entry instead of the 
relative strengths of the nonzero entries of the signal. 

The rest of the paper is organized as follows. Section [TT] 
introduces two coherence parameters of the measurement ma- 
trix and provides a detailed description of OMP. Section [Til] 



gives the main theorems on the performance guarantees of 
OMP for support recovery and signal reconstruction under 
the strong coherence property. Section [IV] prepares for the 
proof and Section [V] proves the main theorems. Finally we 
conclude with discussions in Section [Vll 

II. Two Fundamental Coherence Parameters 

Suppose we are given a measurement vector as y = X/3 + 
77, where X is an n x p unit-norm measurement matrix, j3 £ 
C p is a fc-sparse vector, y £ C" is the measurement and 
t] £ C" is the noise. We define two coherence properties of 
X = [xi, ■ ■ ■ ,x p ] below. The first is worst-case coherence: 

H = max\(xi,Xj)\ = \\X H X - 

which captures the correlation between different columns of 
X. The second is average coherence: 



V 



X — maxl J2(*i,Xi) =-^-r\\( xHx - 1 ) 1 
- 1 1 I rrf. p — 1 



which captures the average correlation between one column 
of X and the remaining columns of X. 

We say a measurement matrix X satisfies the strong 
coherence property if the equation below holds: 



It is known that Gaussian random matrices satisfy the 
strong coherence property with high probability as long as 
n > C((logp) 4 ) ifTTl . Several families of deterministic 
matrices are also known to satisfy the strong coherence 
property, including Gabor frames flOl . Kerdock code sets 
|[T2l . and Delsarte-Goethals code sets |[T2l . Notice that the 
condition on average coherence v < can be achieved 
with essentially no cost via"wiggling", i.e. flipping the signs 
of the columns of X [11|. The "wiggling" procedure doesn't 
change fj, and 1 1 JSC 1 1 2 - 

The goal of this paper is to present performance guarantees 
on OMP for both support recovery and signal reconstruction 
under the assumption that X satisfies the strong coherence 
property, when the measurements are corrupted by noise. In 
particular, two variants of OMP that differ in the way they 
terminate the iterations are analyzed, i.e. Algorithm Q] with 
known sparsity level, and Algorithm [2] with a stopping rule. 

III. Performance Guarantees of OMP 

We define the minimum-to-average ratio MAR and the fth- 
largest-to-average ratio LAR( t ) of the signal f3 respectively as 



MAR 



I 2 ■ 

I mm 



LAR 



(0 



(2) 



where \/3\m is the tth largest absolute value of j3, \/3\ m - m is 
the smallest nonzero absolute value of /?, and k is the sparsity 



Algorithm 1 OMP with a fixed number of iterations 



1: Input: an n xp matrix X, a vector y £ 
level k 

Output: an estimate S of the true model S 
Initialization: So := empty set, residual r$ = y 
for t := 1 : k do 

/ := X H r t . 1 
i := argmaxj 
it := 5 t -i U {]} 



and a sparsity 



end for 

S := S k 



Algorithm 2 OMP with a stopping rule 
1: Input: an n x p matrix X, a. vector y £ C", and a 

threshold level 5 
2: Output: an estimate S of the true model S 
3: Initialization: Sq := empty set, residual ro = y, set the 

iteration counter t = 1 
4: while H-X^rt-iHoo > S do 
5: / := X ff r t _! 
6: i := argmaxj \fj\ 
7: St := S t -i U {3} 

8: n-^y-x^xgXsX'xgy 

9: t:=t+l 

10: end while 

11: S :=S t -i 



level of k. The signal-to-noise ratio SNR and minimum 
signal-to-noise ratio SNR m i n are defined respectively as 



SNR = 



\\m 
n\v\\r 



SNR m i n — 
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A. Performance Guarantee for Support Recovery 

We have the following theorem for OMP with knowledge 
of sparsity level k in Algorithm Q] 

Theorem 1. Suppose X satisfies the strong coherence prop- 
erty for any p > 128, and rj ~ CAf(0,a 2 I). If the sparsity 
level of ft satisfies 



k < min 



4\\ x \\2 1o s.p q/j 2 iogp 



(4) 



for ci = 50^ and C2 = 104^2, and its nonzero entries 
satisfy 



l(t+i) 



> 



2<Ty/(l + a) \ogp 



1 - ci/J V /(fc - t) \ogp 
or write differently, that is 

4(1 + a) f k \ogp 



LAR 



(t+i) 



> 



(l-ci/V(fc-«) logp) 2 V«SNR 



(5) 



(6) 



for < t < k — 1 a«c/ a > 1, then the OMP algorithm 
in Algorithm Q] successfully finds the support of ft with 
probability at least 1 - fc(p Q 7r) _1 - 2p~ 2Iog2 - 4p _1 . 

For the OMP algorithm with a stopping rule in Algo- 
rithm |2j we have the following theorem. 

Theorem 2. If X satisfies the strong coherence property 
for any p > 128, and i] ~ CAf(0, a 2 1). If the sparsity 
level of ft satisfies and its nonzero entries satisfy © for 
a > 1, and choose 5 = cry (1 + a) logp, f/zen f/ze OMP 
algorithm in Algorithm [2] successfully finds the support of ft 
with probability at least 1— (/c+l)(p a 7r) _1 — 2p _21 °e2_4p-i 
in k iterations. 

Since MAR < LAR (t+1) for all < t < k - 1, we 
have the following corollary respectively from Theorem Q] 
and Theorem |2] 

Corollary 3. If X satisfies the strong coherence property for 
any p > 128, and rj ~ CA/"(0, er 2 /). T/'f/ie sparsity level of ft 
satisfies ©, ant/ if satisfies 



MAR > 



4(1 



fc logp 
nSNR 



(7) 



(1 - Ci/Vfc log p) 2 

/or a > 1, f/zen f/ze support of ft is successfully recovered by 
OMP with probability at least 1 - fc^Tr)" 1 - 2p~ 21og2 - 
4p _1 tzs/ng Algorithm [7] one/ wz'f/z probability at least 1 — 
(fc + l) ^"^)" 1 - 2p ~ 21og2 - Ap" 1 by Algorithm |2] vwf/i 
S = o~\J(l + a) logp z'« fc iterations. 



Let = c\[i\Jk logp G (0, 1), then (0 implies the sparsity 
level k satisfies 

(l-9f SNR min 



k < 



logp 



4(1 + a) 
Combining with (0), we have 

n(l -6») 2 SNR 



(8) 



fc < max 

o<e< 
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4(1 + a) logp ' c 2 /! 2 logp' 

ciHxJilogp}' (9) 

where the first term is determined by SNR m ; n , which is signal 
dependent; the second term and the third term are determined 
by the worst-case coherence and the spectral norm of the 
measurement matrix X. If X is a tight frame, \\X\\% = p/n, 
the third term becomes k < 0(n/ logp). From the Welch 
bound /i is lower bounded as fi > O^nT 1 ! 2 ), we write the 
worst-case coherence as p, = Csn^ 1 ' 1 for some C3 > and 
7 > 2. Therefore the maximum sparsity level is determined 
by the second term in (0, yielding k < 0((n/ logp) 2 / 7 ), 
and when 7 = 2 this gives k < 0(n/ logp). In particular, 
the sparsity level k doesn't depend on the profile of signal 
strength of ft, i.e. MAR of the signal. 

We have another corollary on partial recovery. 

Corollary 4. If X satisfies the strong coherence property for 
any p > 128, and r/ ~ CAf(0, a 1 !). If the sparsity level of 



ft satisfies and its nonzero entries satisfy (|6]l for < 
t < k' — I < k — 1 and a > 1, then the OMP algorithm 
in both Algorithm\l\and Algorithm^ successfully selects k' 
entries from the support of ft with probability at least 1 — 
fc'G^Tr)- 1 - 2p- 21og2 - 4p-\ 

It is worth noting that it is not necessarily the support of the 
fc'-largest entries that is recovered from the first k' iterations. 
The next corollary provides the condition on detecting the 
fc'-largest entries. 

Corollary 5. If X satisfies the strong coherence property for 
any p > 128, and rj ~ CAT(0, a 2 1). If the sparsity level of ft 
satisfies ©, and its nonzero entries satisfy 



l(t+i; 



> 



|(t+2) +2v\ / (l + a) logp 
1 - ci ) tt v /(fc - t) logp 



(10) 



for < t < k' - 1 < k - 1 and a > 1, then the OMP 
algorithm in both Algorithm\l\and Algorithm\2\ successfully 
selects k' largest entries from the support of ft with proba- 
bility at least 1 - k'(p a Ti)- 1 - 2p~ 21og2 - 4P" 1 . 

B. Performance Guarantees for Signal Reconstruction 

Furthermore, we could reconstruct the amplitude of the 
signal ft by first reconstructing the amplitude on the detected 
support II via least-squares estimation as 

z = X^y, 

then ft is obtained by filling in the zero entries. We have the 
following theorem. 

Theorem 6. If X satisfies the strong coherence property 
for any p > 128, and rj ^ /V(0,(7 2 /). If the sparsity 
level of ft satisfies and its nonzero entries satisfy (O 
for < t < k — 1 and a > 1, then the £2 norm error of 
the signal reconstructed by least-squares estimation on the 
support recovered by the OMP algorithm satisfies 

\\ft-ft\\l < 4(1 + a)ka 2 logp 

with probability at least 1 - k(p a n)^ 1 - 2p~ 21og2 - 4p _1 
using Algorithm [7J and with probability at least 1 — (fc + 
l)(p Q 7r) _1 - 2p~ 21og2 - Ap- 1 using Algorithm® 

C. Comparison with Other Results 

We now compare our bound with the performance guar- 
antee of OMP for support recovery provided in [7j, which 
we have modified slightly for complex Gaussian noise. In 
order to select exactly the correct support with probability 
at least 1 - (fc + l)^"^) ^ 1 for the OM P Algorithm |2] with 
the stopping rule S = o\J (1 + a) logp, the signal ft needs 
to satisfy 



MAR > 



4(1 + a) 



(l-(2k-lW 



k logp 
n SNR 



(11) 



therefore the sparsity level of j3 satisfies 
'n(l-0) 2 SNR mil 



k < max min 
o<e<i 



4(1 + a) log p 



2/' 



The first term is the same as that in (O, but the second term 
gives k ~ 0{^ v ), therefore k < ©(n 1 / 7 ). We achieved 
a much tighter bound (O by sacrificing the probability of 
success to 1 - (k + l)(p a n)- 1 - 2p~ 21og2 - Ap- 1 . 

We also compare with the performance guarantee of the 
Sorted One Step Thresholding (SOST) algorithm analyzed in 
ifTUl . which outputs the index set of the fc-largest entries in 
absolute values of / = X H y from line[5]in AlgorithmQ] By 
rephrasing Theorem 4 in IflOl , in order to select the correct 
support with probability at least 1 — Qp^ 1 , the sparsity level 
of /3 satisfies 



k < max min \ 

o<e<i I 



n(l - 



SNR„ 



16 \ogp 
-}' 



800/i 2 logp MAR' 
(12) 



21ogp- 

Compare to (O, we see that when X is not a tight frame, it is 
the third term that degrades the performance guarantees. On 
the other hand, the SOST algorithm performs poorly when 
the MAR is of the signal is much smaller than 1, as seen 
from the second term in ( fT2l . 

IV. Preparations for Proof 
A. Statistical Orthogonality Condition (StOC) 

The Statistical Orthogonality Condition (StOC) for a mea- 
surement matrix X is first introduced in iflOl . 

Definition 1. Let IL = (jn , . . . , ir p ) be a random per- 
mutation of {1, . . . ,p\, and define II = . . . , 7Tfc) and 
n c = (7Tfc+i, . . . , 7T p ) for any k < p. Then the matrix X is 
said to satisfy the (fc, e, S)-StOC, if there exist e, 5 G [0, 1) 
such that the inequalities 



||(X#*n-I)*||oo<e|M| a) 



\X^aXuZ\\ 



<e z\ 



(13) 
(14) 



hold for every fixed z € C fe with probability exceeding 1 — 5, 
with respect to EL 

We have the following proposition rephrased from IflOl 
stating that the StOC is satisfied with high probability if X 
satisfies the strong coherence property. 



Proposition 1 ([10]). If the measurement matrix X satisfies 
the strong coherence property, then it satisfies (k, e, S)-StOC 
for k < n/(2 \ogp), with e = lO/i-^/2 logp and 5 < Apr 1 . 

If ( fT3l and ( fT4l hold for a realization of permutation 
fl, then for t < k, let EL; = (tti, . . . , it t ) and iTf = 
(fft+i, ■ ■ ■ , 7Tfc), if ( fl3l l and (fl4l hold for every z £ C k , so 



that n ( unj = n and IE* n 
have 



For every z € C*, we 



Xu t X U c 



XtjcXtIj X^cXftc 



<e z 



L n t ^n t 
from (flJl l. therefore 

\\(XgX nt -It)z\\oo<e\\z\\ 2 , 
\\X^X Ut z\\ x < e\\z\\ 2 . 

Moreover, from (fl4l we have 



\X^cXYl t z\\oo — 



[Xg c Xn t Xfi c X n <i] 



z 










<e z 



B. Conditioning of random submatrices 

We need the following proposition that shows a random 
submatrix of X is well-conditioned with high probability, 
which is essentially due to Tropp 0~3), and first presented in 
the form below by Candes and Plan lfT4l . 

Proposition 2 (£13), 03]). Let ft = {tt\, . . . , n p ) be a ran- 
dom permutation of {1, ... , p}, and define II = (n\ , ... ,77 k) 
for any k < p. Then for q = 2 \ogp and k < pj (4||X||2), we 
have 

(E[||^n-/||^]) 1/9 



< 2 1/q 30/zlogp+ 134 



t 2kpqj}ogp 
p 



(15) 



with respect to the random permutation IE. 

The following proposition lfl~4l states a probabilistic bound 
on the extreme singular values of a random submatrix of X, 
by applying Markov's inequality 

Pi(\\X^X n -I\\ 2 >l/2)<2m[\\XgXn-If 2 ] 
to Proposition [2] 

Proposition 3 ( ITT41 ). Let El = (tt\, . . . ,-k p ) be a random 
permutation of {1, . . . ,p}, and define IE = (m, . . . , 7Tfc) for 
any k < p. Suppose that (i(X) < 1/(240 logp) and k < 
p/(c^\\X\\2logp) for numerical constant c 2 = 104s/2, then 
we have 

Pr X n - I\\ 2 > 1/2) < 2p- 2l °^ 2 . 

Notice that \\XgX n -I\\ 2 = max{A max (X^X n )-l, 1- 
XminiXgXn)}, where A max (X^X n ) and \ min (XgX n ) 
are the maximum and minimum eigenvalues of X^ Xjj, i.e. 
all the eigenvalues of X^Xyi are bounded in [1/2,3/2]. If 
for a realization of permutation EI, \\X^Xyi — I H2 > 1/2, 
we have 

IKXfXn)- 1 !^ <2 and HXn(X^Xn)- 1 1| 2 < V2. 
Moreover, for t < k and IE t = (m, . . . , 7r t ), we have 
\\X*X Ut -Ith>l/2, 



since the eigenvalues of Xff t Xu t are majorized by the 
eigenvalues of X^Xu- 

C. Correlated Gaussian Noise 

Let P <E C nxn be a projection matrix such that P 2 = P. 
Since 77 ~ C/V(0, ct 2 /„) is i.i.d. complex Gaussian noise, 
X H Pr] ~ CAf(0, a 2 X H PX) is also Gaussian distributed, 
but is correlated with covariance matrix a 2 X H PX . We want 
to bound PrdlX P^Hoc > r) for some r > 0. First, we 
need the Sidak's lemma IT5l below. 

Lemma 4 (Sidak's lemma). Let \X\ , ■ ■ • , X n ] be a vector 
of random multivariate normal variables with zero means, 
arbitrary variances a\, ■ ■ ■ , a 2 and and an arbitrary corre- 
lation matrix. Then, for any positive numbers ci, ■ ■ ■ ,c n , we 
have 

n 

Pr(|Xi| < ci, • • • , \X n \ < c n ) > n Pr (l^l < c 0- 

i=l 

Since X H Pr 1 ~ CAf(0, a 2 X H PX), then each xf P77 ~ 
aAf(0,of), where of = a 2 xfPx % < a 2 . Then 

Prflxf P??| < t) = 1- -e- r2/<T ^ > 1 - -e"^^ 2 . 

7T 7T 

Following Sidak's lemma, for r > we have 

v 

PrdlX^PryHoc < r) > IJPrflxf Pry] < r) 

i=l 

> (1- i e " T > 2 )P 
>l-/ e -> 2 , 

7T 

provided the RHS is greater than zero. We have the propo- 
sition below. 

Proposition 5. Let 77 be a random vector with i.i.d. 
C/V(0,<7 2 ) entries, P be a projection matrix, and X be a 
unit-column matrix, then for r > we have 

Pr{\\X H Pri\\ 00 <r)>l-^e-^''', 

provided the RHS is greater than zero. 

Now let t = cry (1 + a) logp for a > 1, we have 

Pr{\\X H P?]\\ oc , < y/a 2 (l + a) logp} > 1 - (p^tt)" 1 . 

V. Proof of Main Results 

We first write the data vector /3 as /3 = Pnz, where z G 
C fe is a deterministic vector, and Pn € M. pxk is a partial 
identity matrix composed of columns indexed by II. Then 
the measurement vector can be written as 

y = X(3 + i] = XP n z + 77 = X n z + 77, 

where Xn denotes the submatrix of X composed of columns 
indexed by II. 



We note that in OMP, the residual n, t = 0, ■ ■ • , k — 1 
is orthogonal to the selected columns in previous iterations, 
so in each iteration a new column will be selected. Define 
a subset lit which contains t variables that are selected at 
the tth iteration. Then P t = Xn^X^Xn^X^ is the 
projection matrix onto the linear subspace spanned by the 
columns of Xji t , and we assume Pq = 0. 

We want to prove II t C II by induction. First at t = 0, 
II t = C II. Assume at iteration t, II t C II, then the residual 
Tt can be written as 

r t = (I- Pt)y = (I - Pt)X n z + (J - P t ) v 4 at + n t . 

Let Mi = ||*£st||oo, = WX&atWo, and N t = 

\\X H n t \\oo, then a sufficient condition for II t+ i C IT, i.e. for 
OMP to select a correct variable at the next iteration is that 

M> - M^c > 2N t , (16) 

since 

WXgrtWoo > Mfi -N t > M^+Nt > WX^nW^. 

A. Bounding and M^ c 
Define the event 

Gi = {X satisfies the (k, e, <5)-StOC}, 

that happens with probability at least 1 — 4p _1 with respect 
to n from Proposition Q] and the event 

g 2 = {\\xi(x n -i\\ 2 < 1/2}, 

which happens at least 1 — 2p~ 21og2 with respect to ft from 
Proposition |3] Let the event Q = Q\ fl G-2- From the above 
discussions the event Q holds with probability at least 1 — 
4p~ J - 2p~ 21og2 with respect to ft. 

Now we bound and M^ c under the event Q. Let 
Ilf = II\II f be the set of yet to be selected indices of the sup- 
port of f3, and /3rp = Zm be the corresponding coefficients. 
Since (I— P t )Xnz <G TZ(Xw) belongs to the linear subspace 
spanned by the columns of Xjjc, we can find a vector w of 
dimension (k — t) such that Xjj^w = (I — P^Xtlz, where 
the vector w can be written as 

w = {X^Xn^X^il - P t )Xncz n c 
= zn% - {X^aXn^y 1 X"aP t X n ^zui ■ 

We need the following lemma on eigenvalue majorization 
from 0. 

Lemma 6 (0). The minimum and maximum eigenvalues of 
Xffc(I — Pt)Xri| are bounded as 

(Xgc(I - P t )X n c) > \ min (X£X n ), 
Amax {X£o{I - P t )X n c) < \ max (XgX n ). 



The readers are referred to Q for the proof. Since we have 

H| 2 < \\(Xg t Xn<)-%\\X« t (I - P t )X nt zn°h (17) 
< 2\\X^Xnh\\z^h < 3||zn=|| 2 , (18) 

where (fTTI i follows from Lemma [6] and ( fT8l follows from 
Proposition [3] Also, 

||Xnc-PtA"n^ n -||oo = \\XgcXu t (Xg t Xn t y 1 Xg t Xw i zn'= t \\ OD 

^eWiXgXnJ-'XgXntZn-h 
<e\\(X»X n r%\\X£X n c\\ 2 \\z n c\\ 2 

therefore can be bounded as 

M*, = HX^Xn^ni - ^P 4 X n ^n ? ||oc 

> ||z nf Hoc - MXgcXn. - I)zui\\oo - \\X^P t Xn^ t \\ 

> ||znc|| 0O -2e||znc|| 2 . (19) 
where (fT9l follows from (TOT i. Next, M^ c can be bounded as 

M^c = \\Xg e (I - P t )X U z\\oo 

= \\x£Xn;w\\oo 

< e||w||a < 3e||2fnf|| 2 . (20) 
where (|20l follows from (fT4t . 

fi. Bounding N t 

Conditioned on the event Q, for each P t , since / — P t is 
also a projection matrix, define the event 

Ht={N t <cry/(l + a)logp}, t = (),••• ,k- 1. (21) 

Then from Proposition |5j it happens with probability at 
least 1 — (p a Tr)^ 1 with respect to 77. We further define 
the event % = PQZq%u then from the union bound 
P-r(H\G) = Pr(U) > 1 - fc^Tr) -1 . Similarly, for the event 
H' = ^UqHu then from the union bound Pt(H'\G) = 
Pr(H')>l- (k + l)(p a ir)-\ 

C. Proof of Theorem [7J and Theorem [2] 

Define the event X = Q n %, from the above discussions 
we have Pr(I) > 1 - k(p a n)- 1 - 2p~ 21o s 2 - 4p-\ Now we 
are ready to analyze the OMP performance under the event 
Q. We want to prove Il t C II by induction. 

Substituting the bounds 4T9jl, (f20]l and (EJ into (O, it is 
sufficient that at the fth iteration 

II || 00 

> 5e||z n? || 2 + 2ay/(l + a)logp. (22) 

Note that ||zn°||oc > lkn=|| 2 < VA: - *|/%+i)> 

(1221 is satisfied by 

|0| (t+ i) > 5eVfc - t|%+i) + 2ay/(l + a) log p. 

Since fc < l/(c}/i 2 logp) and e = 10p^/2\ogp, this is 
equivalent to the condition in © for < t < k — 1, therefore 



a correct variable is selected at the fth iteration, II t C II. 
Since the sparsity level of j3 is k, the OMP algorithm in 
Algorithm[T]successfully finds the support of j3 in k iterations 
under the event X, and we have proved Theorem Q] 

Now we define the event X 1 = %' fl Q, where X' C X 
which happens with probability at least 1 — (k + l)(p a 7r) _1 — 
2p- 21o g 2 — 4p _1 . Conditioned on the event X', in order to 
prove Theorem|2] we need to further show that H-X^rtHoo > 
5 for < t < k — 1 so that the algorithm doesn't stop early, 
and \\X H 

^™fc || 00 — $ so that the algorithm stops at the /cth 
iteration. While the latter is obvious from the definition of 
Hk, for the first inequality we have 

H^rtHoo > - N t 

> \\zn-\\oo - 2e||zn t H| 2 - ^(1 + a) logp 

(23) 

o . 

>3ej|z nf !|2 + o-V(l + a)logP (24) 

> a^(l + a) logp = S, 

where d23l follows from ( TT9l and (f2lT> . and (1241 follows from 
(|22|. 

Remark: The proof of Corollary [4] is straightforward by 
early-terminating the induction procedure at the k'th iteration. 

D. Proof of Corollary\5\ 

Again we prove by induction. First at t = 0, II t = C II. 
Assume at iteration t, the OMP algorithm has successfully 
detected the t largest entries of \/3\. For i 6 ITJ that 
corresponds to the t + 1th largest entry of |/3|, we have 

\xf r t \ > \zi\ - 2e\\zni h - oy/{l + a)\ogp, 

= \P\(t+i) - 2e||e n f ||a - o-^(l + a)\ogp 

from a simple variation of (Tf9l . On the other hand, for j £ IIj 
that corresponds to the rest undetected entries of /?, we have 

\xfr t \ < \ Zj \ + 2e||z n c|| 2 + + a) logp, 

< |/3| (t+2) + 2e||2fnj||a + ^^/(l + a) log p. 
To detect the t + 1th largest entries it is sufficient to have 

- |/% +2) > 4eV 7 ^ r 7|/?| (t+ i) + 2,7^(1 + a) logp. 

This is satisfied when dTOb by simply plugging it into the 
above equation. 

E. Proof of Theorem \6\ 

Conditioned on the event that the support is successfully 
recovered, since 

z = xy = X^(X n z + r)) = z+ (X^Xn)- 1 X^ V , 

where z = /3n is the non-zero part of j3, and 

^uxgxnr'wiwx^wi 

< ikWXgnWl, < 4(1 + a)ka 2 logp, (25) 



it follows that 

||/3 - P\\l = \\2-z\\l< 4(1 + a)ka 2 logp. 

VI. Conclusion 

In this paper, we provide coherence-based performance 
guarantees of two variants of OMP for both support recovery 
and signal reconstruction of sparse signals when the measure- 
ments are corrupted by noise. It is shown that if X satisfied 
the strong coherence property, then with n > O(fclogp), 
OMP recovers a fc-sparse signal with high probability. In 
particular, the guarantees obtained here separate the proper- 
ties required of the measurement matrix from the properties 
required of the signal. The resilience of OMP to variability 
in relative strength of the entries of the signal might be an 
advantage in applications like multi-user detection in wireless 
communications because it makes power control less critical 
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